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Polling  Questions 


Polling  Question  1 :  Give  me  a  sense  of  how  many  of  you  are  coming  to  this 
webinar  from  the  perspective  of  the  CMMI  Risk  Management  Process  Area: 

O  I  have  completed  an  official  Introduction  to  CMMI  course. 

O  I’m  involved  in  doing  risk  management  according  to  CMMI,  but  I  haven’t  taken 
Introduction  to  CMMI. 

o  I’m  here  because  I’m  interested  in  risk  management  and  identifying  risks,  but  not  in 
CMMI. 

O  None  of  the  above/other. 

Polling  Question  2:  I’d  like  to  know  whether  you  have  risk  management 
processes  in  place  in  your  program  or  organization,  and  --  if  so  --  how  “mature” 
you  consider  it  to  be: 

o  We  have  had  risk  management  in  for  some  time  --  it’s  well  established  and  pretty 
mature. 

o  We  have  risk  management  in  place,  but  it’s  fairly  new  and  not  too  well  established, 
o  We  don’t  have  risk  management  processes  in  place  yet. 


— Software  Engineering  Institute  Carnegie  Mellon 


Identifying  Program  Risks 
11  December  2008 
©2008  Carnegie  Mellon  University 


VO,  P3 


CMMI  Risk  Management  (RSKM)  Process  Area 


Programs  Often  Don’t  Document  Real  Risks 


■  Independent  Technical  Assessments  (ITAs)  and  SCAMPI  appraisals 
probe  interviewees  for  top  risks  to  the  program. 

■  The  “top  risks”  named  by  the  interviewees  often  can’t  be  found  in  the 
programs  risk  repository. 

■  A  risk  repository  that  is  visible  to  all  is  usually  more  political  than 
useful. 

■  There  is  often  no  real,  repeatable  process  in  a  program  for  identifying 
risks. 
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Developing  Good  Risk  Identification  Techniques 
Took  Time 


■  Interviews  and  workshops  with  DOD  program  managers  in  the  early 
1990s  pin-pointed  risk  identification  as  the  biggest  need. 

■  This  became  the  first  focus  of  the  SEI  risk  program. 

■  40  field  tests  were  conducted  with  a  broad  range  of  software 
developers  before  coming  up  with  a  good  interviewing  technique  for 
risk  identification  (the  “Taxonomy-Based  Risk  Identification”  method). 

■  Several  more  years  passed  before  the  Condition-Consequence  risk 
statement  form  and  the  “Threshold  of  Success”  were  incorporated 
into  the  identification  approach. 
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Threshold  of  Success  (ToS) 


■  What  is  it? 

The  minimum  set  of  conditions  that  MUST  be  met  for  your  project 
members  to  consider  the  project  a  marginal  “success” 

-  What  does  it  have  to  do  with  risk  statements? 

Identified  risk  are  evaluated  against  the  conditions  itemized  in  the 
program’s  ToS 

Mapping  risks  to  ToS  conditions  assures  that  appropriate  attention  is 
being  paid  to  the  risks  with  the  biggest  impact  on  the  project’s  view  of 
success 

■  When  do  I  need  one? 

^  Any  time  you  want  to  identify  risks  or  categorize,  evaluate,  and  prioritize  a 
list  of  risks 

Reevaluated  when  a  major  event  takes  places  that  has  a  direct  impact  on 
the  project 
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How  to  Build  a  ToS 


Define  a  minimum  set  of  conditions  the  MUST  be  met  for  your  project 
members  to  consider  it  a  “Success” 

Make  sure  these  goals  are  specific,  measurable  and  set  for  a  certain  time 

in  the  future  (typically  the  end  of  the  project)  and  that  they  are  attainable  and 
realistic 

Guidelines 

■  Put  yourself  at  the  end  of  the  project 

■  Build  yourself  a  picture  of  failure 

■  List  those  things  it  would  take  for  your  project  to  fail 

■  Example:  We  did  not  meet  “must  have”  requirements,  We  didn’t  deliver  on  schedule 
etc’ 

■  Convert  those  into  Threshold  of  Success  statements  (we  MUST  do  X  or  have  shown 
that  our  product  has  met  at  least  Y  to  reach  our  ToS) 

■  Stay  within  one  slide  limit  with  4-5  items  on  it 
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Threshold  of  Success  (ToS)  —  An  Example 


■  Approval  to  continue  the  project  at  Key  Decision  Point  4  (KDP4)  in 
late  fall,  1999. 

■  Support  from  {Gov’t  Agency}  management  to  maintain  the  2001 
budget  at  not  less  than  $7M. 

■  People  in  the  field  are  “begging”  for  {product}  deployment  (because 
they  see  that  it  will  make  their  tasks  so  much  easier),  and 

■  A  single  point  of  entry  for  all  logistic  data  (i.e.,  resolution  of  all  issues 
with  other  responsibility  areas  within  the  {Gov’t  Agency}  of  where 
data  comes  from,  what  is  the  most  reliable  source  of  information, 
who  will  enter  it,  etc.) 
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The  Condition-Consequence  Risk  Statement 


Condition 


The  requirements  for  passing 
Milestone  B  have  not  yet  been 
defined 


the  current  schedule  and  cost 
estimates  may  be  inadequate. 
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The  Condition-Consequence  Form  Supports  Analysis 


Timeframe 


Impact 


Probability 


The  requirements  for  passing 
Milestone  B  have  not  yet  been 
defined 


the  current  schedule  and  cost 
estimates  may  be  inadequate. 
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The  Risk  Statement 


A  “standard”  format  for  risk  statements  provides: 

■  clarity 

■  consistency 

■  a  basis  for  future  risk  processing 


Condition 


A  good  Risk  Statement  is 
»*  fact-based 
»*  actionable 
»♦  brief 
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A  Simple  Risk  Identification  Interview 
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Risk  Statements  from  lnterviews1 


■  Lack  of  executive  sponsorship  (maybe  because  of 
change  in  the  Administration);  time  delays,  frustrations, 
credibility,  and  morale,  and  [a  department  co¬ 
sponsoring  the  project]  may  pull  out  of  [the  project]. 

■  The  majority  of  software-to-software  interfaces  are  not 
defined  &  controlled;  incomplete  interfaces  results  in 
no  benefits  from  [the  project]. 

■  There  has  been  inadequate  schedule  discipline 
(milestones,  slippage,  monitor  progress,  good  project 
management)  on  this  project;  with  no  intervention  the 
project  will  continue  to  slip  &  slide. 
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Risk  Statements  from  lnterviews2 


■  Unstable  resources;  leads  to  inability  to  plan. 

■  [The  project]  doesn't  have  a  formal  priority  for 
resources;  project  slippage  and  waste  of  resources. 

■  "Shared  Vision",  but  lack  of  agreement  above  & 
between  agencies  in  means  to  reach  objective; 
marginal  utility  of  product  and  waste  of  resources. 
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Risk  Statements  in  the  “If-Then”  Form 


■  It’s  too  easy  to  start  thinking  up  worries  about  the  far  future  not  based 
on  anything  going  wrong  today  (“If  our  code  gets  written  badly,  then 
our  product  might  not  be  accepted  by  customers”) 

■  It’s  too  easy  for  higher  management  to  reject  the  underlying  issue  out 
of  hand. 

■  People  often  masquerade  their  own  desires  for  how  the  program 
should  be  managed  as  a  “risk  statement”;  for  example,  “If  we’re  not 
able  to  hire  enough  programmers  to  do  the  development  work  by 
July,  then  we  could  deliver  the  product  late.” 
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For  Further  Reading  —  SEI  Publications  on  Risk  Identification 


-  CMU/SEI-93-TR-6,  “Taxonomy-Based  Risk  Identification,”  M.J.  Carr  et  al 

-  CMU/SEI-94-TR-014,  “A  Construct  for  Describing  Software  Development 
Risks,”  D.  Gluch 

-  Continuous  Risk  Management  Guidebook,  August  1996,  A.J.  Dorofee  et  al 

(available  only  online  at  http://www.sei.cmu.edu/publications/books/other-books/crm.guidebk.html) 

-  CMU/SEI-99-TR-029,  “SRE  Method  Description  (Version  2.0)  &  SRE  Team 
Members  Notebook  (Version  2.0),  R.C.  Williams  et  al 

-  CD-ROM  for  CMU/SEI-99-TR-029,  “Interviewing  Process”  (available  free  of  charge 
from  SEI  Customer  Relations  -  just  call!) 

■  CMU/SEI-77-TN-777,  “Mini  Software  Risk  Evaluations:  Going  Light 
Evaluating  Risks  in  Software  Intensive  Projects,”  R.C.  Williams  and  G.  Taran 
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Setting  the  Stage  for  Good  Risk  Identification 


■  It  appears  to  be  commonly  believed  that  the  only  possible  form  of  a 
risk  repository  is  one  that  contains  all  the  program  risks  and  is 
accessible  to  all  program  members. 

■  Some  believe  CMMI  and  the  SEI  expect  this. 

■  In  practice,  this  is  the  surest  way  to  leave  all  the  most  important  risks 
undocumented. 

■  An  undocumented  risk  can  get  lost  to  everyone  --  far  better  to  have 
risks  documented  privately  than  not  to  be  documented  at  all. 

■  If  program  management  expects  openness  from  “worker  bees”  and 
middle  management  about  the  risks  they  see,  program  management 
must  likewise  be  open  about  their  risks. 
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A  Risk  Repository  can  be  Distributed  and  Locally  Private 


Organization 


other 

programs 
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Summary 


■  A  work  team  identifying  risks  needs  to  agree  on  an  end-point  against  which 
to  identify  and  analyze  the  risks. 

■  There  needs  to  be  a  standard  way  of  capturing  (documenting)  a  risk. 

-  Facilitators  need  practice  to  become  comfortable  writing  risks  in  front  of  a 
group. 

-  There  are  many  ways  for  program  management  to  support  good  risk 
identification: 

Encourage  documentation  of  risks  privately  at  the  working  team  level 

Integrate  risk  identification  and  management  into  normal  project  management 

Accept  any  risk  identified  into  the  repository  -  don’t  “vet  them  out” 

Acknowledge  that  the  program’s  decision-makers  are  the  real  “risk  managers,” 
and  have  the  decision-makers  step  up  to  the  job 
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Related  SEI  Courses 


There  are  public  offerings  of  the  SEI’s  Continuous  Risk  Management 
(CRM)  course  in  2009  on  the  following  dates 


March  3-4  (Pittsburgh) 
June  23-24  (Pittsburgh) 
October  21-22  (DC) 


There  are  public  offerings  of  the  SEI’s  Systems  Acquisition  Survival 
Skills  (SASS)  course  -  which  includes  heavy  emphasis  on  risk 
management  --  on  the  following  dates 

April  15-17  (Pittsburgh) 

July  21-23  (Pittsburgh) 

November  18-20  (DC) 
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MSCE  Risk  Management  Offerings 


The  Mission  Success  in  Complex  Environments  (MSCE)  initiative  at 
the  SEI  has  the  following  offerings  available  as  individual  client 
engagements: 

Risk  Management  Tutorial  -  A  one-half  day  tutorial  that  provides  an 
overview  of  practical,  success-driven  risk  management  concepts. 

Risk  Management  Workshop  -  A  one-day  workshop  that  presents  a 
practical  risk  management  framework  and  teaches  how  to  assess  a  risk 
management  capability  against  the  framework. 

Risk  Management  Clinic  -  A  two-day  workshop  focused  on  improving 
an  organization’s  risk  capability  to  achieve  practical,  effective 
management  of  risks  to  program  objectives. 


Contact  customer-relations@sei.cmu.edu  for  more  information 
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A  Related  Podcast 


"Culture,  Psychology,  and  Motivation:  Getting 
Program  Decision-Makers  to  Use  and  be  Part  of 
Risk  Management  Processes" 

A  panel  session  presentation  recorded  live  at  the  2007 
INCOSE  International  Symposium 


http://www.sei.cmu.edu/programs/acquisition-support/rw_w_bumpers.mov 
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Software  Engineering  Institute  Carnegie  Mellon 
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